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ABSTRACT 


Technological  advances  have  made  atmospheric  mesoscale  modeling  at 
very  fine  resolutions  readily  available  to  a  great  number  of  organizations. 

Though  initial  operational  results  show  some  skill  with  respect  to  synoptic  scale 
forecasts,  many  of  the  problems  associated  with  mesoscale  error  growth  and 
predictability  have  been  ignored.  Understanding  mesoscale  error  is  critical  to 
accurately  interpreting  mesoscale  model  results  and  output  from  tactical  decision 
aids  (TDA’s). 

This  study  examines  mesoscale  error  growth  and  predictability  through 
controlled  numerical  model  experiments.  A  known  “true”  atmosphere  is  created 
through  the  use  of  the  US  Navy’s  Coupled  Oceanographic/Atmospheric 
Mesoscale  Prediction  System  (COAMPS).  Virtual  observations  are  randomly 
sampled  from  this  atmosphere  to  provide  data  for  ingest  into  forecasts  using  the 
NCAR/Penn  State  MM5  mesoscale  model.  Forecast  results  for  ten  cases  are 
compared  against  the  “true”  atmospheric  solution  and  error  statistics  are 
calculated  for  wind  speed  and  geopotential  height  fields.  Results  show  how  error 
growth  and  predictability  are  affected  by  different  variables  such  as  boundary 
conditions,  weather  regime,  sample  size  and  sample  distribution.  A  scale 
separation  of  error  is  also  performed  in  order  to  assess  the  impact  of  synoptic 
scale  error  on  mesoscale  error. 

Results  indicate  that  error  growth  and  predictability  are  dependent  on  all 
the  above  mentioned  variables.  Domain  size  and  location  define  the 
wavelengths  of  error  that  are  allowed  to  grow  while  boundary  conditions  limit 
which  error  grows  within  the  domain.  Contrary  to  previous  studies,  errors  did 
grow  on  the  mesoscale  within  a  24  hour  forecast  cycle.  Scale  separation  results 
show  that  mesoscale  error  is  dominated  by  synoptic  sale  error. 
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I.  INTRODUCTION 


The  vast  number  of  technological  breakthroughs  in  the  last  decade  have 
allowed  for  enormous  growth  and  opportunity  in  the  field  of  mesoscale  modeling. 
The  increase  in  computer  speed  and  power  and  the  widespread  availability  of 
quality  computing  hardware  has  continued  to  fuel  the  vision  that  we  can  and 
should  predict  our  environment  at  the  finest  resolutions  imaginable.  This  growth 
has  also  engendered  in  some  minds  a  belief  that  the  finer  the  resolution,  the 
better  the  forecast.  While  the  recent  growth  in  mesoscale  modeling  and 
increased  resolutions  has  led  to  some  useful  insights  into  the  key  physical 
properties  that  govern  mesoscale  dynamics  (Doyle  1997,  Thompson  et  al.  1997), 
the  general  question  of  predictability  and  the  influence  that  the  synoptic  scale 
has  on  the  predictability  limit  of  the  mesoscale  has  been  largely  misunderstood 
or  ignored. 


A.  THE  OPERATIONAL  SIGNIFICANCE  OF  PREDICTABILITY 

The  operational  significance  of  the  general  predictability  of  mesoscale 
models  and,  in  particular,  specific  mesoscale  features  cannot  be  overlooked.  A 
good  understanding  of  these  concepts  is  crucial  to  effectively  using  today’s 
mesoscale  model  generated  fields.  The  US  Navy  and  other  services  are 
increasingly  relying  on  the  accuracy  of  their  mesoscale  models  not  only  to 
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produce  a  detailed  mesoscale  forecast  but  also  provide  accurate  information  into 
sophisticated  tactical  decision  aids  (TDA’s).  The  output  of  these  TDA’s  can  give 
detailed  guidance  to  the  warfighter  who  is  trying  to  assess  the  environment 
he/she  will  be  working  in,  but  very  rarely  gives  him/her  a  good  picture  of  the 
reliability  of  that  guidance.  The  model  information  is  passed  to  the  TDA  and 
assumed  to  be  accurate  and  relatively  error  free,  yet  there  is  no  indication  given 
that  a  particular  mesoscale  feature  may  not  even  be  “predictable”  at  the  temporal 
or  spatial  scale  of  the  operation.  While  a  good  forecaster  can  and  should 
accurately  assess  the  character  of  the  error  in  a  particular  model  forecast 
through  experience  and  comparison  with  other  sources,  there  is  currently  no 
means  by  which  a  TDA  can  identify  error.  Further  uncertainty  in  the  guidance  is 
brought  about  by  the  fact  that  very  little,  if  any,  actual  mesoscale  observations 
are  assimilated  into  the  mesoscale  model.  Thus,  we  must  rely  on  the  accuracy 
of  the  synoptic  scale  initial  conditions  to  pass  their  dynamically  significant 
features  down  to  the  mesoscale.  At  what  scale  then  must  we  routinely  observe 
the  atmosphere  in  order  to  accurately  forecast  mesoscale  weather  events?  And 
how  accurate  must  we  be  on  the  synoptic  scale  in  order  to  achieve  a  desired 
degree  of  accuracy  on  the  mesoscale?  Finally,  how  does  the  variability  in 
sample  size  and  location,  mesoscale  structure  pattern,  and  boundary  conditions 
affect  error  growth  and  ultimately  predictability?  It  is  with  these  questions  in 
mind  that  we  approached  this  work,  with  the  ultimate  desired  result  being  a 
better  understanding  of  error  growth  and  predictability  on  the  mesoscale. 
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B.  WHAT  IS  PREDICTABILITY  THEORY? 

The  bulk  of  predictability  work  in  the  past  two  decades  has  been 
accomplished  by  Lorenz  (1 982, 1 993)  and  Anthes  (1 984).  Lorenz’  work  focused 
on  the  concept  of  the  predictability  time  limit,  which  represents  the  span  of  time 
between  the  best  estimate  of  the  state  of  the  atmosphere  based  on  initial 
observations  and  an  estimate  of  its  state  at  a  future  time.  The  predictability  time 
limit  represents  the  point  where  the  forecast  becomes  essentially  unusable  due 
to  inherent  numerical  model  forecast  errors.  Beyond  the  predictability  time  limit, 
the  likelihood  of  verifying  the  forecast  is  no  better  than  random  guesswork.  The 
best  estimate  for  the  state  of  the  atmosphere  at  the  initial  time  depends  first  on 
the  quality  of  the  observational  sample  and  then  on  the  method  of  data 
assimilation.  Based  on  Lorenz'  reasoning,  the  better  this  estimate,  the  longer 
the  time  before  the  predictability  time  limit  is  reached.  Figure  1  shows  a 
hypothetical  change  in  numerical  model  forecast  error  (NMFE)  over  time  and 
highlights  the  impact  initial  error  has  on  predictability.  At  the  initial  time,  the 
NMFE  is  small  but  grows  rapidly  due  to  the  lack  of  dynamic  consistency  between 
the  initial  model  state  analyzed  by  observations  and  the  dynamic  allowances  set 
forth  in  the  model  physics.  As  the  model  spins  up,  this  initial  state  develops  into 
a  state  that  is  more  consistent  with  the  models  physical  equations  and  thus  a 
relative  minimum  occurs  in  the  forecast  error.  Further  development  in  the 
simulation  shows  that  forecast  error  continues  to  grow  due  to  the  incompleteness 
of  the  model’s  physical  equations,  its  discretization,  and  truncation  error.  At 
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some  point,  the  NMFE  reaches  a  critical  level  where  the  model  essentially  has 
no  skill  -  the  predictability  time  limit. 

The  critical  level  of  NMFE  can  be  defined  in  various  ways.  In  global 
atmospheric  models,  the  critical  NMFE  is  defined  as  the  error  variance  (mean 
square  of  the  difference  of  some  variable)  associated  with  two  randomly  chosen 
atmospheric  states,  or  twice  the  climatological  error  variance  (Anthes  1986). 
From  the  perspective  of  the  warfighter,  however,  the  critical  level  of  NMFE  is 
dependent  on  the  nature  of  the  operation  which  requires  the  numerical  guidance, 
and  may  in  fact  be  much  lower.  A  third  possible  measure  of  the  predictability 
time  limit  is  the  use  of  the  standard  deviation  about  the  mean  of  a  particular 
model  variable.  Though  more  a  measure  of  the  forecast  skill  of  a  particular 
solution,  this  measure  also  gives  us  insight  into  the  predictability  of  the  solution. 
If  the  root  mean  square  error  of  two  numerical  model  solutions  beginning  with 
small  differences  at  the  initial  time  exceeds  the  standard  deviation  about  the 
mean  for  a  particular  solution,  then  the  other  solution  has  in  effect  reached  its 
predictability  time  limit  with  respect  to  the  first  solution.  By  calculating  the  mean 
of  a  certain  variable  such  as  wind  speed  in  a  control  environment  and  then 
comparing  the  RMS  error  of  a  particular  numerical  solution  to  the  standard 
deviation  about  the  mean  of  the  control,  one  can  gain  an  accurate  measure  of 
the  skill  of  that  particular  forecast  solution.  If  the  RMS  value  exceeds  the 
standard  deviation  about  the  mean  of  the  control,  then  in  essence,  the  forecast 
has  no  skill,  and  simply  taking  the  mean  winds  across  the  domain  would  provide 
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a  better  forecast.  We  have  utilized  this  third  means  of  measuring  predictability 
for  all  the  error  growth  comparisons  in  this  study. 

A  similar  predictability  limit  to  the  time  limit  can  also  be  applied  in  space 
and  is  called  the  predictability  spatial  limit.  It  represents  the  spatial  scale  at 
which  small  scale  variations  become  essentially  randomly  distributed  and  beyond 
which  there  is  little  useful  information  upon  which  to  base  a  forecast.  The 
distinction  between  the  two  predictability  limits  is  that  forecasts  become  less 
meaningful  for  increasing  time  scales,  whereas  they  become  less  meaningful  for 
decreasing  spatial  scales.  Thus,  as  mesoscale  modeling  efforts  seek  to  define 
increasingly  smaller  spatial  scales  over  increasingly  longer  time  scales,  it 
becomes  clear  that  understanding  the  predictability  of  certain  features  becomes 
even  more  important. 

The  question  of  a  theoretical  predictability  limit  becomes  more  complex  in 
that  it  often  differs  for  different  mesoscale  phenomena.  Although  previous 
research  suggested  that  predictability  decreases  rapidly  with  scale  to  limit 
mesoscale  predictability  to  less  than  a  day,  common  experience  with  mesoscale 
models  shows  that  this  is  not  always  the  case  and  that  practical  predictability 
time  limits  are  often  much  longer.  Baumhefner  (1984)  showed  that  for  the 
synoptic  scale,  predictability  decreases  with  decreasing  scale  at  a  typical  error 
doubling  time  of  about  two  days.  Tennekes  (1978)  suggested  that  the 
mesoscale  would  reach  the  predictability  time  limit  faster  than  the  synoptic  scale 
due  to  the  transfer  of  energy  from  the  small  to  larger  scales  by  three  dimensional 
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turbulence.  However,  for  the  mesoscale  phenomena  that  do  not  fit  the  average 
spectrum  of  atmospheric  turbulence,  the  rate  of  this  transfer  of  energy  and  thus 
forecast  error,  may  differ  considerably.  Warner  (1992)  also  suggested  that  some 
mesoscale  phenomena,  in  particular  those  forced  by  fixed  features  such  as  a 
coastal  mountain  range,  might  be  more  predictable.  Shukla  (1984)  found  that 
since  atmospheric  variability  is  less  in  low  lattitudes  than  in  middle  latitudes,  the 
limit  to  deterministic  predictability  is  shorter  in  the  tropics  than  in  the  mid¬ 
latitudes,  provided  the  error  growth  rates  for  a  particular  feature  are  the  same. 
These  results  were  all  primarily  obtained  by  performing  predictability  experiments 
on  global  models.  This  thesis  will  seek  to  examine  and  confirm  a  few  of  the 
results  from  Anthes,  Baumhefner  and  Shukla’s  experiments  but  also  seeks  to 
uncover  some  of  the  recent  nuances  of  error  growth  on  the  mesoscale  by 
utilizing  current  mesoscale  model  capabilities  in  a  self-contained  numerical 
modeling  experiment. 


C.  HYPOTHESES 

This  thesis  studies  the  dependence  of  mesoscale  error  growth  and 
predictability  on  several  different  factors.  Consequently,  we  suggest  that  the 
initial  error  growth  as  well  as  the  predictability  in  a  numerical  forecast  is  strongly 
influenced  by  four  factors: 


6 


(1)  The  observational  sample  size,  scale  and  distribution. 

(2)  The  method  of  data  assimilation  used  to  project  information  and  error 
from  observations  onto  the  forecast  model. 

(3)  The  choice  of  boundary  conditions  and  frequency  at  which  they  are 
applied. 

(4)  The  characteristic  structure  of  the  mesoscale  regime. 

Regarding  factor  (1),  we  seek  to  discover  at  what  scale  we  must  routinely 
observe  the  state  of  the  atmosphere  in  order  to  accurately  forecast  mesoscale 
weather  events.  Regarding  factor  (2),  we  seek  to  understand  how  sensitive 
mesoscale  forecasts  are  to  both  synoptic  and  mesoscale  uncertainty  in  the  initial 
conditions  for  a  particular  data  assimilation  system.  Regarding  factor  (3),  we 
seek  to  understand  how  boundary  conditions  impact  the  character  of  the  error 
growth  at  all  domain  sizes.  Finally,  regarding  factor  (4),  we  seek  to  understand 
how  the  predictability  time  limit  changes  for  different  mesoscale  regimes.  Our 
specific  hypotheses  are  that: 

1.  Variation  in  the  observational  sampling  pattern  at  the  synoptic 
scale  leads  to  variation  in  analyses  that  directly  impact  error 
growth  and  predictability  in  mesoscale  forecasts. 
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2.  For  a  limited  area  model,  the  choice  of  boundary  conditions  and 
domain  size  directly  impacts  the  character  of  the  error  growth  by 
restricting  the  wavelength  of  allowable  solutions. 

3.  Error  growth  on  the  mesoscale  correlates  with  error  growth  on 
the  synoptic  scale  although  errors  grow  more  rapidly  at  smaller 
scales. 

4.  Mesoscale  predictability  is  weather  regime  dependent  and  error 
tends  to  be  larger  in  scale  and  more  variable  in  weather  regimes 
with  greater  kinetic  energy. 


D.  EXPECTED  RESULTS 

The  expected  results  of  this  thesis  are  in  essence  a  better  understanding 
of  error  growth  and  predictability  as  it  applies  to  the  mesoscale.  We  will  show 
that  normal  variations  in  observational  sampling  size  and  distribution,  model 
horizontal  grid  spacing,  as  well  as  the  system  of  data  assimilation  and  boundary 
conditions  all  impact  error  growth  and  thus  predictability.  We  will  also  estimate 
the  shape  of  the  NMFE  growth  curves  as  a  function  of  these  variables.  By 
defining  the  critical  NMFE  amount  as  the  standard  deviation  about  the  mean  of 
our  control  atmosphere,  we  will  evaluate  the  forecast  skill  and  the  predictability 
limit  for  different  mesoscale  regimes,  hopefully  showing  that  model  forecasts 
initialized  from  better  observational  sampling  regimes,  and  thus  better  analyses, 
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will  extend  the  predictability  time  limit  for  most  mesoscale  features.  We  also 
expect  that  at  horizontal  resolutions  where  there  are  little  or  no  observations,  the 
predictability  time  limit  will  be  much  shorter  and  mesoscale  forecasts  more 
suspect. 

Finally,  we  expect  our  results  will  produce  more  questions  as  we  uncover 
the  nature  of  the  relationship  of  various  factors' to  error  growth  on  the  mesoscale. 
Practically  speaking,  we  will  never  be  able  to  fully  observe  the  atmosphere  at  the 
scales  at  which  we  desire  to  predict.  The  question  that  remains  is  then  one  of 
understanding  and  managing  the  error  growth  that  we  are  certain  will  exist  at 
those  scales.  By  better  understanding  how  synoptic  scale  variations  impact  the 
mesoscale  domain,  we  may  be  able  to  develop  techniques  to  limit  these 
variations  or  at  the  very  least,  be  able  to  identify  when  a  mesoscale  forecast  is 
suspect  and  when  it  may  be  the  right  tool  for  the  job. 
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II.  APPROACH 


One  of  the  difficulties  of  performing  atmospheric  predictability  studies  is 
the  inability  to  completely  sample  the  entire  atmosphere  and  thereby  have  a  well- 
defined  reference  grid  with  which  to  perform  the  studies.  Since  operational 
observational  sampling  does  not  often  capture  true  mesoscale  structure,  doubt  is 
cast  upon  the  accuracy  of  the  error  calculations  for  specific  points  within  the 
domain.  This  study's  approach  therefore,  utilized  the  benefits  of  a  controlled 
environment  using  numerical  model  experiments.  Although  described  in  more 
detail  in  the  following  sections,  the  basic  approach  of  our  study  follows. 

Instead  of  relying  on  operational  sampling  to  describe  a  particular 
mesoscale  regime,  an  idealized  atmosphere  was  generated  from  a  mesoscale 
model  forecast.  These  model  grids  were  then  taken  as  the  "true"  atmospheric 
conditions  for  the  course  of  the  experiments  providing  the  advantage  of  knowing 
the  structure  of  the  atmosphere  at  every  point  on  the  grid.  Virtual  observations 
were  then  developed  by  randomly  sampling  the  ideal  atmosphere  in  order  to  help 
assess  the  impact  of  observational  sampling  differences  on  both  analyses  and 
forecast.  By  combining  the  virtual  observations  with  a  synoptic  scale  first  guess 
field,  we  performed  several  mesoscale  model  runs  via  a  separate  model  in  order 
to  produce  forecast  fields  for  comparison.  We  could  then  determine  the 
characteristic  error  in  the  model  as  well  as  the  shape  of  the  NMFE  curves  by 
comparing  the  analysis  and  forecast  results  to  the  “true”  control  initial 
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atmosphere.  This  procedure  was  performed  for  two  separate  weather  patterns 
(winter  and  summer),  and  two  distinct  model  predicted  fields  (geopotential 
height)  and  wind  speed.  Other  sampling  strategies  included  varying  the  number 
of  observations  in  the  analyses  as  well  as  running  an  experiment  with  no  virtual 
observations,  producing  an  analysis  and  forecast  from  initial  guess  points  only. 

To  understand  the  effects  of  boundary  conditions  on  error  growth,  we  also 
performed  identical  forecast  experiments  using  two  separate  boundary 
conditions.  In  one  experiment  we  utilized  the  Navy  Operational  Global 
Atmospheric  Prediction  System  (NOGAPS)  as  the  boundary  conditions  since  this 
simulated  the  conditions  actually  used  in  Navy  operations  today,  while  in  the 
other  we  utilized  the  "ideal"  atmospheric  conditions  at  the  boundary  to  give  us  a 
sense  of  the  "best"  case  forecast  error  growth.  Finally,  in  order  to  understand 
how  error  might  be  passed  down  from  one  scale  to  another,  we  performed  a 
scale  separation  of  error  for  a  single  random  case  run.  Throughout  the  testing 
phase,  the  main  criteria  for  assessing  the  impact  of  the  various  sampling 
approaches  on  the  forecast  error  growth  was  the  change  in  forecast  skill  and  the 
predictability  time  limit  as  described  by  the  numerical  model  forecast  error 
curves.  Table  1  below  summarizes  the  various  experiments  performed  with  the 
created  data  sets. 


12 


Summary  of  Experiments 

Goal 

Desiqn 

1.  Determine  how  varying  sample  size 
affects  error  growth. 

Compared  analyses  of  experiments  using  0, 

100,  200,  and  400  point  samples  to  the  "true” 
atmosphere  and  calculated  RMS  error.  Noted 
trend  in  error  with  increasing  point  samples. 

2.  Determine  how  various  random  distributions 
of  observations  affect  error  growth. 

"Sampled"  random  observations  from  the  "true" 
initial  atmosphere,  producing  analysis  and 
forecast  for  ten  separate  cases.  Calculated 
RMS  errors  for  wind  speed  at  500mb  and  noted 
the  error  growth  differences  between  the 
various  numerical  solutions. 

3.  Determine  how  error  growth  is  affected  by 
varying  mesoscale  regimes 

Calculated  error  growth  statistics  for  a  definitive 
winter  and  summer  pattern. 

4.  Determine  how  mesoscale  error  is  affected 
by  larger  scale  error. 

Performed  a  scale  separation  of  error  by 
projecting  larger  scale  fields  onto  4km  grid  and 
then  calculated  the  associated  scale  error. 

5.  Determine  how  choice  of  boundary 
conditions  affects  error  growth. 

Ran  identical  forecast  simulations  for  both 
winter  and  summer  regimes  using  two  different 
sets  of  boundary  conditions. 

Table  1.  Summary  description  of  various  experiments  contained  within  the 
study. 


A.  “IDEAL”  ATMOSPHERE  SPECIFICATION 

The  idealized  mesoscale  atmosphere  used  as  the  basis  for  all  the 
experiments  and  the  random  observations  was  created  by  running  the  US  Navy's 
Coupled  Ocean/Atmosphere  Mesoscale  Prediction  System  (COAMPS)  at  40 
vertical  levels  for  a  36  hour  "observing"  period  using  four  nested  domains  having 
horizontal  grid  spacings  of  108,  36,  12,  and  4  km  centered  over  the  San 
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Francisco  and  Monterey  Bay  region  (Figure  2).  A  12  hour  data  assimilation 
forecast  was  run  for  the  12  hours  prior  to  the  time  period  of  initialization  in  order 
to  spin  up  reasonable  mesoscale  horizontal  and  vertical  structure  along  with 
dynamically  consistent  cloud  and  rain  fields.  The  initial  conditions  for  the  data 
assimilation  and  the  outermost  domain's  lateral  boundary  conditions  originated 
from  the  2.5  degree  NOGAPS  analyses  for  the  corresponding  time  period. 

The  experimental  forecast  fields  used  for  comparison  to  the  “true” 
atmosphere’s  forecast  were  created  using  the  NCAR/Penn  State  MM5 
mesoscale  model.  The  model  contained  30  vertical  levels  and  was  run  with  full 
physics.  MM5  was  chosen  as  the  forecast  model  in  order  to  allow  for  unique 
model  solutions  and  to  prevent  reintroduction  of  error  specific  to  the  COAMPS 
model.  Initial  analyses  on  12  vertical  levels  were  created  with  NOGAPS  first 
guess  fields,  and  the  randomly  selected  observations  from  the  COAMPS  “truth” 
mentioned  above.  The  12  level  analyses  were  then  interpolated  to  the  30 
vertical  levels  to  initialize  the  MM5  forecasts. 

It  should  be  noted  that  in  order  to  provide  a  control  forecast  using  MM5, 
the  original  COAMPS  “true”  atmosphere  at  reduced  vertical  resolution  but  with 
identical  horizontal  grid  points  was  used  for  initialization.  Since  the  experimental 
forecasts  were  run  with  MM5  starting  from  a  12  vertical  level  analysis  (as  in 
NOGAPS)  ,  the  MM5  control  forecast  also  had  to  contain  12  vertical  levels  to 
provide  fair  estimates  of  error  that  were  independent  of  the  differing  vertical 
resolutions.  Although  the  control  forecast  contained  less  vertical  structure  than 
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the  original  COAMPS  “true”  atmosphere,  the  horizontal  resolutions  remained  the 
same.  The  control  forecast  thus  represented  the  best  prediction  that  MM5  could 
achieve  based  on  the  COAMPS  “true”  atmosphere  at  consistent  vertical 
resolution  and  was  therefore  a  good  basis  for  comparison  to  the  experimental 
forecast  runs.  Figure  3  is  a  schematic  diagram  which  summarizes  the 
experiment’s  design. 


B.  RANDOM  “OBSERVATIONS” 

In  order  to  provide  consistent  amounts  of  observations  for  the  initialization 
of  the  forecast  runs,  we  utilized  a  random  number  generator  to  select  random 
observations  from  the  108  km  COAMPS  “true”  atmosphere.  Ten  cases, 
representing  ten  separate  seed  numbers  into  the  random  number  generator, 
were  run  to  provide  some  statistical  significance  to  the  study.  100  observations 
were  selected  randomly  for  each  case  run  except  for  the  sample  size  studies 
where  we  selected  200  and  400  observations  respectively.  The  same  seed 
numbers  used  in  the  winter  case  were  also  used  for  the  summer  case  in  order  to 
provide  consistency  for  comparison  between  the  two  weather  regimes.  The 
forecast  runs  were  thus  initialized  with  NOGAPS  global  fields  and  the  randomly 
selected  observations  added.  Figure  4  shows  the  random  distribution  of 
observations  for  a  specific  seed  number  with  100,  200  and  400  observations 
respectively. 
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C.  THREE  DIMENSIONAL  MULTIQUADRIC  INTERPOLATION 


Three  dimensional  multiquadric(3DMQ)  interpolation  was  used  as  the 
data  assimilation  method  for  these  experiments  due  to  its  relative  simplicity  and 
the  researcher’s  familiarity  with  the  method.  An  added  advantage  of  using  the 
3DMQ  approach  was  that  it  had  the  ability  to  resolve  features  at  the  smallest 
scales  that  were  represented  by  observations  in  a  particular  region  of  the 
analysis  domain  (Nuss  and  Titley  1994).  This  ensured  that  we  could  accurately 
calculate  the  error  associated  with  the  finest  mesoscale  features. 

3DMQ  uses  radial  basis  functions  which  assume  isotropic  structure 
functions  for  the  environment  (Nuss  and  Titley  1994).  This  approach  differs  from 
the  anisotropic  error  covariance  structure  functions  of  Optimal  Interpolation  (01) 
and  three  dimensional  variational  analysis  (3DVAR)  schemes.  These  data 
assimilation  schemes  rely  on  prior  knowledge  of  the  true  state  of  the  atmosphere 
in  order  to  give  an  optimal  analysis  based  on  an  estimated  error  covariance 
function.  However,  error  in  the  estimates  of  the  observation  and  background 
error  covariance  functions  strongly  impact  the  accuracy  of  the  optimal  fit.  3DMQ 
interpolation  is  not  affected  by  errors  in  covariance  estimates  since  it  does  not 
rely  on  prior  knowledge  of  the  true  state  of  the  atmosphere.  Theoretically  then, 
there  would  be  smaller  potential  error  variations  from  one  weather  regime  to 
another,  allowing  for  consistent  analysis  between  different  weather  regimes, 
which  particularly  suited  the  dual  weather  pattern  approach  of  this  experiment. 
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D.  WEATHER  REGIMES 


The  idealized  mesoscale  atmosphere  was  generated  for  two  distinct 
weather  regimes,  1)  a  deep  extratropical  cyclone  with  an  associated  front 
interacting  with  the  coastline  occurring  on  February  6th,  1998,  and  2)  a  summer 
high  pressure  pattern  which  produced  a  typical  sea  breeze  pattern  in  the 
Monterey  Bay  region,  occurring  on  August  25th,  1997.  Figures  5[a-e]  and  6[a-e] 
show  the  progression  of  each  weather  pattern  for  the  24  hour  forecast  period  for 
the  “true”  atmosphere.  Of  primary  importance  in  the  differing  weather  regimes  is 
the  mesoscale  structure  associated  with  each  regime  and  the  amount  of  kinetic 
energy  that  each  system  contains.  For  instance,  we  expect  that  on  the  4km 
nest,  error  growth  associated  with  the  winter  pattern  would  be  primarily 
influenced  by  the  passage  of  the  cold  front  and  the  model’s  correct  timing  and 
orientation  of  this  event.  For  the  summer  pattern,  however,  we  expect  the  error 
growth  on  the  4km  nest  to  be  associated  more  with  the  background  flow  and  with 
the  correct  representation  of  the  marine  planetary  boundary  layer.  Due  to  the 
greater  amounts  of  energy  associated  with  the  winter  pattern,  we  also  expect 
that  the  magnitude  of  the  errors  associated  with  this  system  will  be  greater  than 
those  in  the  summer  pattern.  Another  factor  to  consider  is  the  start  time  of  the 
models  for  the  two  weather  regimes.  The  winter  pattern  begins  at  00Z  and  thus 
the  first  12  hours  of  the  forecast  are  in  the  “cold”  part  of  the  forecast  cycle.  The 
summer  regime  however,  started  at  12Z  and  thus  in  this  case,  the  first  12  hours 
of  the  forecast  are  in  the  “hot”  part  of  the  forecast  cycle. 
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1 .  The  Winter  Regime 

Figure  5a  shows  the  winter  pattern  beginning  at  00Z  on  6  February,  1998. 
A  987mb  surface  low  pressure  system  is  located  in  the  Gulf  of  Alaska  with  an 
associated  baroclinic  frontal  zone  extending  to  the  southeast.  The  low  has 
begun  occluding  already  at  this  point  and  a  wave  is  forming  along  the  frontal 
boundary  at  approximately  35N,  130W,  which  later  develops  into  a  new  low 
pressure  system.  South  of  the  wave,  the  frontal  band  extends  to  the  southwest 
and  is  located  approximately  450  nautical  miles  from  the  Monterey  Bay,  which 
encompasses  the  majority  of  the  4km  domain.  Surface  winds  in  the  vicinity  of 
the  Monterey  Bay  are  southerly  at  25kts.  At  500mb,  the  108km  domain  is 
dominated  by  a  significant  long  wave  trough  whose  axis  lies  between  135W  and 
140W,  notably  behind  the  surface  front.  At  00Z,  a  vorticity  maximum  is  entering 
the  base  of  the  trough  and  has  an  associated  1 05kt  jet  maximum.  The  front  left 
quadrant  of  the  vorticity  maximum  corresponds  with  the  developing  wave  at  35N, 
130W. 

By  six  hours  into  the  forecast,  the  Gulf  of  Alaska  low  pressure  system  has 
occluded  and  moved  off  to  the  northwest.  The  surface  front  extending  from  the 
new  low  pressure  center  at  35N,  127W  is  now  about  300  nautical  miles  (nm) 
away  while  surface  winds  near  Monterey  have  begun  to  turn  to  the  southwest 
and  have  increased  to  30kts.  At  the  500mb  level,  the  vorticity  maximum  has 
begun  to  progress  through  the  base  of  the  trough  and  has  taken  on  a  more 
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northwesterly  orientation.  The  trough  axis  is  located  at  approximately  130W,  still 
behind  the  surface  front.  The  jet  maximum  has  reduced  in  speed  to  90  kts. 

Figure  5b  shows  the  development  of  the  pattern  twelve  hours  into  the 
forecast.  The  surface  front  has  entered  the  4km  domain  and  has  made  landfall 
near  the  California/Oregon  border,  though  it  is  still  approximately  50  nm  away 
from  Monterey.  Surface  pressures  in  the  vicinity  of  Monterey  have  decreased  by 
8-1  Omb  over  the  12  hour  period  and  surface  winds  have  continued  turning  to  the 
west  while  maintaining  speed  at  30  kts.  The  low  pressure  center  is  now  located 
at  37N,  125W,  just  off  the  northern  California  coast.  The  upper  level  pattern 
shows  that  the  vorticity  maximum  has  continued  to  translate  through  the  base  of 
the  trough  and  both  the  trough  axis  and  the  vorticity  maximum  are  nearly  in  the 
same  location  as  the  surface  front.  The  jet  maximum  has  remained  relatively 
constant. 

Sometime  between  12  and  18  hours  into  the  forecast,  a  significant  event 
occurs  which  eventually  is  reflected  in  the  error  growth  curves  for  the  winter 
pattern.  During  this  time  frame,  the  surface  front  enters  the  western  portion  of 
the  4km  domain  and  makes  landfall  in  the  vicinity  of  Monterey.  By  18  hours 
however,  the  surface  front  has  passed  through  Monterey  and  the  other  side  of 
the  4km  domain  and  surface  winds  have  shifted  completely  to  the  west  and 
northwest.  The  low  pressure  center  has  continued  to  move  to  the  northeast  and 
is  now  located  over  the  California/Oregon  border  with  a  central  pressure  of 
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996mb.  At  the  500mb  level,  the  vorticity  maximum  has  completely  passed 
through  the  base  of  the  trough  indicating  the  trough  has  stopped  deepening. 

Figure  5c  shows  the  final  progression  of  the  winter  pattern  24  hours  into 
the  forecast.  The  entire  4km  domain  and  the  majority  of  the  other  domains  are 
under  the  influence  of  the  subsidence  behind  the  cold  front  and  the  atmosphere 
has  become  relatively  stable  again.  At  the  500mb  level,  the  vorticity  maximum 
has  moved  inland  a  considerable  distance  while  the  trough  has  filled.  The  jet 
maximum  has  also  decreased  to  70  kts  and  is  now  located  well  to  the  south  of 
California. 

2.  The  Summer  Regime 

Figures  6(a-c)  show  the  24  hour  forecast  for  the  summer  pattern 
beginning  at  12Z  on  25  August,  1997.  Unlike  the  winter  regime,  the  summer 
pattern  remains  fairly  dynamically  consistent  throughout  the  forecast  cycle.  The 
synoptic  scale  is  dominated  by  an  upper  level  trough  to  the  north  in  the  Gulf  of 
Alaska  that  is  also  reflected  at  the  surface  as  a  filling  low  pressure  system,  and 
to  the  south  by  a  weak  high  pressure  ridge.  There  is  a  weak  stationary  front  that 
separates  the  two  systems  that  also  does  not  move  considerably  during  the  24 
hour  forecast. 

The  primary  influence  on  the  4km  domain  is  the  presence  of  the  weak 
high  pressure  ridge  extending  from  just  north  of  Pt.  Conception  to  the  southwest. 
The  associated  surface  winds  around  the  high  set  up  a  weak  onshore  flow  and 
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favorable  conditions  for  the  development  of  thermally  forced  sea  and  land 
breezes  in  the  Monterey  Bay  area.  Although  this  case  is  a  bit  climatalogically 
atypical  for  the  summer  considering  the  location  and  strength  of  the  trough  at  the 
coast,  it  was  chosen  for  its  stark  contrast  to  the  spatially  and  temporally  dynamic 
winter  pattern. 


E.  NUMERICAL  MODELING  FORECAST  ERROR  (NMFE) 

Thus  far,  we  have  only  alluded  to  the  general  concept  of  Numerical 
Modeling  Forecast  Error,  its  existence,  and  its  impact  on  forecast  accuracy.  How 
this  error  is  defined  specifically,  however,  depends  on  many  factors.  This  study 
will  in  part  show  how  NMFE  varies  as  a  function  of  weather  regime,  sample  size 
and  distribution,  boundary  conditions,  and  horizontal  model  grid  spacing.  Since 
generating  root-mean-square  (RMS)  error  statistics  was  the  simplest  method  for 
calculating  NMFE  with  time,  we  utilized  this  approach  as  the  primary  method  for 
determining  the  NMFE.  We  also  calculated  the  bias  errors  for  selected  model 
parameters.  The  drawback  associated  with  the  simple  RMS  approach  to  NMFE 
is  that  it  gave  little  insight  into  the  source  of  the  error;  whether  it  was  related  to 
location,  timing  or  intensity.  To  compensate  for  this  shortcoming,  we  attempted 
to  categorize  the  errors  by  other  means  in  order  to  better  quantify  the  NMFE  for 
the  above-mentioned  factors.  Another  factor  to  consider  was  the  fact  that  the 
area  of  study  was  located  in  a  coastal  environment  where  RMS  errors  were  not 
uniformly  spread  across  the  domain  but  were  rather  correlated  with  the 
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topographic  effects  of  the  coastline.  Thus,  further  error  growth  studies 
performed  over  a  more  uniform  domain  may  show  different  results. 

For  consistency  in  comparison  throughout  the  experiments,  we  chose  to 
calculate  the  NMFE  using  geopotential  heights  and  wind  speed  and  direction  at 
500  mb.  Although  500mb  is  less  influenced  by  mesoscale  forcing  at  the 
surface,  this  level  was  found  to  be  representative  of  model  error  at  all  levels.  We 
also  calculated  the  average  error  throughout  the  column.  Again,  the  character  of 
the  error  between  the  various  levels  was  fairly  consistent  and  thus  choosing  a 
single  level  to  represent  error  growth  was  representative  of  the  actual  error 
growth  throughout  the  column.  Finally,  the  specific  definition  of  the  critical  NMFE 
with  regard  to  the  model  horizontal  grid  spacing  was  the  standard  deviation 
about  the  mean  of  the  chosen  model  parameter  over  the  given  domain. 


F.  SCALE  SEPARATION  OF  ERROR 

To  gain  a  better  understanding  of  the  effects  of  synoptic  scale  error  on  the 
mesoscale,  we  performed  a  scale  separation  of  the  error  unique  to  each  scale  of 
the  nested  grid.  The  nonlinear  nature  of  atmospheric  processes  allows  energy 
exchange  among  all  scales  of  motion  (Anthes  1984)  and  thus  uncertainty  or  error 
in  any  scale  will  eventually  contaminate  all  scales.  The  design  of  the  nested 
grids  we  have  chosen  for  our  study,  however,  are  one-way  nested  and  thus  error 
can  only  propagate  down  scale.  In  other  words,  error  on  the  4km  domain 
consists  of  error  inherent  to  that  scale  as  well  as  error  propagated  in  from  outer 
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scales  while  error  on  the  108  km  domain  consists  only  of  error  unique  to  that 
scale.  We  utilized  the  following  equation  to  represent  the  total  error  for  the  4km 
grid, 


^total  =  ^108km  ^36km  ^12km  ^4km 

where  the  total  error  is  the  summation  of  the  error  unique  to  each  individual 
scale.  This  same  equation  could  be  applied  to  the  coarser  grids  as  well  by 
simply  dropping  the  error  term  for  the  finer  grid  spacings.  Thus,  the  total  error  at 
12km  would  be 

Htotal  =  £l08km  +  ^36km  +  £l2km 

and  likewise  for  other  nests. 

The  first  task  was  to  represent  each  of  the  larger  scale  fields  (108km, 
36km,  12km)  for  both  the  "true"  atmosphere  and  the  random  case  atmospheres 
on  the  4km  grid.  Figure  7  shows  the  results  of  this  procedure  at  the  twelve  hour 
forecast  for  a  single  winter  regime  case.  The  4km  domain  representation  of  the 
larger  scale  wind  fields  are  each  shown  on  the  4km  domain.  As  expected,  the 
108km  representation  shows  the  long  wave  pattern  while  the  other  three 
representations  show  increasing  amounts  of  mesoscale  structure.  By 
subtracting  the  true  atmosphere’s  4km  representation  of  each  scale  from  the 
particular  case  atmosphere's  4km  representation,  we  could  systematically 
calculate  the  amount  of  error  at  the  4km  grid  scale  that  was  due  to  other  scales. 
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G.  TERMINOLOGY 


To  avoid  confusion  when  referring  to  various  model  forecast  runs,  the 
following  terminology  and  notation  is  used  throughout  the  thesis: 

•  COAMPS  “truth”  refers  to  the  original  ideal  atmosphere  created  for  the 
purposes  of  random  observational  sampling, 

•  MM5  “control”  forecast  refers  to  the  best  representation  of  the  “true” 
atmospheric  progression  through  time  and  is  used  as  the  basis  for 
comparison  in  all  experimental  forecasts. 

•  980206/00Z  refers  to  the  winter  pattern  while  the  summer  pattern  is 
970825/1 2Z. 

•  Analyses  performed  with  COAMPS  will  be  represented  as  simply 
“Analysis”  while  the  initial  forecast  time  for  the  MM5  control  forecast  (also 
an  analysis)  will  be  represented  by  F00. 

•  All  other  forecast  times  will  be  represented  in  the  same  manner  (i.e.F06, 
F12,  F24,  etc.).  Thus,  980206/00zF12  will  be  the  winter  pattern,  00Z  run, 
12-hour  forecast. 

•  For  each  weather  pattern,  we  performed  ten  random  case  runs  where  we 
selected  a  seed  number  for  the  random  number  generator  which  then 
randomly  selected  “observations”  on  the  outer  domain.  Each  case  is 
represented  by  its  unique  seed  number.  For  example,  C148954  is  the 
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case  whose  seed  number  was  148954.  The  same  seed  numbers  were 
used  when  sampling  the  winter  and  summer  cases  for  consistency. 
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III.  FINDINGS 


The  broad  scope  of  this  project  provided  many  avenues  for  exploration 
into  the  character  of  error  on  the  mesoscale  and  the  number  of  different  ways  to 
analyze  the  data  was  staggering.  It  was  immediately  apparent  from  the  resulting 
error  growth  calculations  that  each  of  the  previously  mentioned  factors  impacted 
the  error  growth  in  different  ways,  individually  as  well  as  in  combination.  Figures 
8[a-d]  and  9[a-d]  show  the  complete  500mb  wind  speed  error  growth  curves  for 
all  ten  random  cases  at  all  domains  for  both  weather  patterns,  and  both 
boundary  conditions.  A  cursory  look  at  the  differences  between  the  various  plots 
shows  that  differences  in  weather  regime,  boundary  condition  and  scale  all 
impact  the  character  of  the  error  growth  as  well  as  the  skill  and  predictability  of 
the  forecasts.  In  some  cases,  these  differences  affect  individual  solutions  and  in 
other  cases,  the  set  of  solutions  as  a  whole.  The  findings  for  the  specific  error 
analyses  performed  will  be  presented  here  with  a  more  in-depth  discussion  of 
the  results  presented  in  the  following  chapter. 


A.  THE  EFFECT  OF  SAMPLE  DISTRIBUTION  ON  ERROR  GROWTH 

Each  random  case  of  selected  observations,  when  used  in  the 
initialization  of  a  new  model  run,  produced  a  distinctly  different  solution.  Thus, 
within  the  confines  of  a  particular  weather  pattern  or  boundary  condition,  the 
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error  growth  and  predictability  of  each  solution  was  strictly  dependent  on  how 
well  that  particular  set  of  observations  characterized  the  structure  of  dynamically 
significant  atmospheric  features. 

1 .  Correlation  between  Analyses  and  Forecast  Error 

The  concept  that  a  better  analysis  will  produce  a  better  forecast  is  one  of 
the  reasons  for  maximizing  the  amount  of  information  by  which  a  particular 
mesoscale  model  is  initialized.  This  thought  is  generally  well  accepted  in  the 
mesoscale  modeling  community  and  thus  it  also  follows  that  if  there  is  relatively 
low  amounts  of  error  at  the  analysis,  there  will  be  comparatively  less  error  in 
successive  forecasts  (Anthes  1986).  Our  study  found,  however,  that  while  this 
may  be  the  case  experientially,  the  error  growth  characteristics  of  the  various 
cases  show  otherwise.  Figures  10  - 13  are  scatter  plot  diagrams  of  500mb  initial 
and  24  hour  forecast  wind  speed  errors  for  all  cases  in  both  weather  regimes, 
using  both  sets  of  boundary  conditions.  Table  2  below  lists  the  summarized 
correlation  coefficients  for  the  same  scatter  plots,  but  only  for  the  108km  and 
36km  domains.  The  values  for  the  12km  domain  are  not  listed  since  they  did  not 
accurately  show  the  true  correlation  between  the  initial  and  forecast  error.  The 
temporal  separation  between  F00  and  F24  combined  with  the  small  domain  size 
essentially  meant  that  we  were  correlating  two  distinctly  different  parts  of  the 
weather  pattern.  As  a  result,  the  correlation  coefficients  showed  a  large  negative 
correlation  which  did  not  fit  with  the  positive  correlation  coefficients  of  the  larger 
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scales.  The  values  for  the  4km  domain  are  also  not  available  since  the  forecasts 
all  initialized  from  the  same  first  guess  fields  with  no  observations  present  within 
the  innermost  domain. 


V. 

Winter 

COAMPS  B.C. 

Winter 

NOGAPS  B.C. 

Summer 
COAMPS  B.C. 

Summer 
NOGAPS  B.C. 

108km 

0.54382 

0.03931 

0.10863 

0.30328 

36km 

0.65788 

0.386276 

0.04164 

-0.01080 

Table  2.  Correlation  Coefficients  between  F00  and  F24  forecast  500mb  wind 
speed  errors  (all  cases). 

Probably  the  first  fact  to  notice  is  that  there  is  not  a  great  deal  of 
correlation  in  general  at  any  scale  for  either  weather  regime  or  boundary 
condition.  This  result  might  be  expected  due  to  the  random  nature  of  the  various 
solutions  and  supports  the  argument  for  ensemble  forecasting  which  seeks  to 
take  advantage  of  the  random  nature  of  various  model  solutions  by  perturbing 
initial  conditions  in  its  ensemble  members.  The  result  also  suggests,  however, 
that  a  low  error  at  initial  forecast  time  does  not  guarantee  a  low  error  at  F24, 
even  though  many  forecasters  use  this  rule  of  thumb  when  analyzing  initial 
forecast  fields. 

Secondly,  these  results  also  show  that  there  is  a  greater  amount  of 
correlation  between  initial  error  and  forecast  error  when  the  the  MM5  “control” 
boundary  conditions  are  used  which  suggests  that  the  boundary  conditions  play 
a  significant  role  in  restricting  the  growth  of  various  solutions  away  from  their 
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initial  dynamic  states.  Additionally,  the  results  show  that  there  is  generally  more 
correlation  between  initial  error  and  forecast  error  during  the  the  more 
synoptically  forced  winter  pattern.  This  result  also  supports  the  impact  the 
boundary  conditions  have  on  the  forecast  error  of  the  mesoscale  since  the 
boundary  conditions  tend  to  act  more  upon  those  types  of  weather  patterns  that 
are  synoptically  forced. 

Though  not  as  useful  due  to  the  fact  initial  fields  for  all  ten  cases  were  the 
same  at  the  4km  scale,  the  4km  domain  correlation  measurements  also  present 
some  interesting  findings.  For  both  weather  regimes  and  boundary  conditions, 
there  appeared  to  be  a  grouping  of  solutions  vice  a  natural  spread  as  might  be 
expected  (Figure  13).  Unlike  the  random  spread  seen  at  the  larger  scales,  this 
result  does  not  bode  well  for  ensemble  forecasting  techniques  at  mesoscale 
domain  sizes.  Of  course,  we  must  take  into  account  that  the  representative 
sample  of  solutions  (10  per  weather  regime)  is  fairly  small  and  larger  data  sets 
may  prove  otherwise. 

2.  Asymptoting  Error  Values 

Following  the  reasoning  that  a  better  analysis  produces  a  better  forecast, 
another  reasonable  conjecture  is  that  variance  in  the  initial  error  for  the  ten  case 
solutions  would  remain  somewhat  constant  throughout  the  forecast  and  would 
match  the  variance  of  the  solutions  at  F24.  This  however,  was  not  the  case. 
Figures  8[a,b]  and  9[a,b]  show  that  particularly  at  the  108  and  36km  domains, 
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the  initial  widespread  difference  in  the  various  solutions  eventually  disappear  as 
the  forecasts  progress  and  that  by  F24,  there  is  little  difference  in  the  error 
between  the  various  solutions.  In  addition,  the  magnitudes  of  the  errors  in  these 
cases  seem  to  asymptote  to  a  particular  value,  suggesting  that  there  may  be 
some  constraint  on  the  wavelengths  at  which  the  error  is  allowed  to  grow  for 
these  domain  sizes. 


B.  THE  EFFECT  OF  SAMPLE  SIZE  ON  ERROR  GROWTH 

1.  Increasing  Sample  Size 

As  expected  for  both  the  winter  and  summer  cases,  when  the  number  of 
sample  observations  was  increased,  the  RMS  errors  in  geopotential  height 
decreased  proportionally.  Figure  14  shows  the  benefit  of  increasing  the  number 
of  observations  in  the  analysis  of  a  particular  case.  For  the  winter  pattern,  the 
largest  benefit  is  obtained  when  we  the  increase  the  number  of  observations  to 
200  observations  from  100  observations,  although  analysis  error  continues  to 
decrease  when  400  observations  are  used.  Conversely,  the  benefit  gained  in 
the  summer  pattern  is  more  consistent  between  100,  200,  and  400  observations. 
The  rate  of  RMS  error  decrease  was  roughly  the  same  at  all  levels  although 
there  were  a  few  instances  where  a  larger  increase  was  found  in  the  lower  levels 
of  the  atmosphere. 
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Figures  15[a-d]  and  16[a-d]  show  the  impact  on  NMFE  when  the 
respective  analyses  for  100  and  200  point  samples  are  used  to  initialize  the  MM5 
model  forecasts.  The  400  point  sample  was  not  used  to  initialize  MM5  due  to 
analysis  problems  with  some  variables.  The  three  curves  on  each  plot  show  the 
500  mb  error  growth  for  case  c148954  with  100  points,  the  growth  curve  when 
200  points  are  used,  and  the  standard  deviation  about  the  mean  wind  speed,  by 
which  we  are  measuring  forecast  skill.  The  results  showed  that  the  increased 
observations  impacted  the  error  growth  differently  for  each  type  of  weather 
pattern,  boundary  condition,  and  domain  size. 

a)  Impact  by  wea ther  regime 

Using  the  increased  number  of  observations  seemed  to  decrease 
the  error  for  most  scales  more  in  the  winter  than  in  the  summer,  regardless  of 
which  boundary  condition  was  used.  For  the  summer  regime,  there  was  almost 
no  benefit  at  all  at  the  108km  domain  (Figure  16a).  This  result  suggests  that  the 
synoptically  driven  forcing  of  the  winter  pattern  benefitted  more  from  the  added 
number  of  points  since  it  was  better  able  to  define  the  synoptic  scale  features. 
Since  the  synoptic  features  were  not  as  crucial  during  the  summer,  the  benefit 
was  not  as  great,  particularly  at  108km.  There  is  some  question  about  whether 
the  improvement  gained  by  increasing  the  number  of  observations  in  the  winter 
pattern  is  significant.  The  answer  to  this  question  must  take  into  consideration 
the  specific  requirements  of  the  operations  requiring  the  forecast  information. 
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b)  Impact  by  boundary  condition 

For  the  winter  case,  the  MM5  “control”  boundary  conditions 
provided  more  benefit  than  when  the  NOGAPS  boundary  conditions  were  used. 
For  the  summer  pattern,  there  was  little  difference  between  the  boundary 
conditions,  neither  of  which  seemed  to  have  a  significant  impact.  A  particularly 
interesting  result  was  that  both  sets  of  boundary  conditions  for  both  weather 
patterns  seemed  to  negatively  impact  the  error  growth  of  the  200  sample  point 
case  when  they  were  applied  at  F12.  At  the  larger  scales(108  and  36km), 
applying  the  boundary  conditions  seemed  to  increase  error  relative  to  the  100 
point  sample,  while  at  the  smaller  scales  (12  and  4km),  the  200  sample  point 
case  actually  had  greater  error  than  the  100  point  case!  This  was  particularly 
evident  for  the  summer  pattern  on  the  4km  domain  (Figure  16d).  This  plot 
suggests  that  at  least  for  this  case,  the  initial  benefit  in  the  forecast  gained  by 
adding  more  observations  is  erased  when  the  boundary  conditions  are  applied. 
Similar  experiments  with  the  other  cases  and  different  weather  patterns  may 
prove  otherwise.  It  should  be  noted  that  the  influence  of  the  boundary  conditions 
should  also  decrease  as  one  goes  to  the  inner  domains  . 

c )  Impact  by  domain  size 

Comparing  the  impact  of  the  increased  sample  size  across  the 
domain  sizes  showed  that  for  the  winter  pattern,  the  decrease  in  error  was  about 
the  same  while  for  the  summer  pattern,  there  was  a  greater  benfit  gained  at  the 
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smaller  scales.  The  character  of  the  error  at  the  larger  domains  remained 
basically  the  same  between  the  100  and  200  sample  point  cases. 

It  should  be  noted  that  at  the  4km  domain  for  both  weather 
patterns,  the  standard  deviation  about  the  mean  wind  speed  of  the  control  is 
lower  than  the  error  associated  with  the  forecasts  (see  Figures  15d  and16d). 
Thus  in  essence,  these  forecasts  have  almost  no  skill.  At  this  scale,  the  benefit 
of  increasing  the  number  of  observations  is  related  to  the  skill  or  predictability  of 
the  forecasts.  For  the  summer  regime  (Figure  16d),  the  100  sample  point  case 
has  no  initial  skill  and  only  gains  skill  around  FI 2.  The  200  sample  point  case 
however,  has  skill  from  about  F04  to  about  FI  6  when  the  error  growth  curve 
exceeds  the  standard  deviation  curve.  The  winter  regime  (Figures  15d)  is  even 
less  promising  as  it  essentially  has  no  skill  at  any  time  in  the  forecast. 

These  results  point  to  the  fact  that  increasing  the  number  of  sample  points 
at  initialization  of  a  mesoscale  model  does  not  always  guarantee  less  forecast 
error  at  the  smallest  nest,  but  does  generally  reduce  error  at  larger  scales. 

2.  NOGAPS  Firstguess  Forecast 

Another  experiment  we  performed  related  to  sample  size  was  to  run  the 
forecast  model  without  any  observations.  The  input  into  the  model  would  thus 
only  be  driven  by  the  NOGAPS  first  guess  fields  and  the  MM5  model  physics. 
The  results  of  this  comparison  are  shown  in  Figure  17[a,b]  for  the  winter  case 
using  the  MM5  “control”  forecast  as  the  boundary  conditions.  As  expected,  the 
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forecast  error  associated  with  a  solution  containing  no  observations  was  much 
higher  than  those  cases  that  were  initialized  with  observations,  particularly  at  the 
108  and  36km  domains  (Figure  17a).  However,  this  was  not  true  at  the  inner 
nests  of  12  and  4km.  Figure  17b  shows  the  500mb  RMS  wind  speed  error  for 
the  winter  regime  at  the  12km  and  4km  domains.  Note  that  the  firstguess 
solution  shows  no  real  skill  at  these  domain  sizes  at  any  time  in  the  forecast 
whereas  it  does  show  relative  skill  at  the  108  and  36km  domains.  Note  also  that 
for  the  first  seven  hours  of  the  forecast,  the  NOGAPS  firstguess  solution  has 
comparable  errors  to  the  solutions  that  were  initialized  with  observations.  By  F08 
however,  the  firstguess  solution  dramatically  increases  in  error  and  remains  high 
throughout  the  rest  of  the  forecast  cycle.  Thus,  synoptic  scale  observations 
seem  to  have  little  benefit  in  the  initial  hours  of  a  mesoscale  forecast  at  the 
smallest  nests  of  the  mesoscale  model. 


C.  THE  EFFECT  OF  LATERAL  BOUNDARY  CONDITIONS  ON  ERROR 
GROWTH 

One  of  the  more  surprising  results  of  the  various  experiments  we 
performed  was  the  impact  that  varying  boundary  conditions  had  on  the  character 
of  the  error  growth.  In  his  early  studies  of  predictability  of  a  limited-area  model, 
Anthes  (1986)  suggested  that  by  supplying  accurate  lateral  boundary  conditions 
(LBCs)  on  the  outer  domain  throughout  a  forecast,  the  error  growth  at  the 
mesoscale  would  be  limited.  This  result  was  realized  when  we  used  the  MM5 
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control  forecast  as  the  boundary  conditions  for  the  MM5  forecasts.  Figure  18 
shows  the  500mb  RMS  wind  speed  error  growth  (or  rather  decay)  curves  for  the 
winter  and  summer  synoptic  patterns  at  the  108km  scale  when  the  MM5  control 
forecast  was  used  as  the  boundary  condition.  The  downward  trend  in  error  is 
contrary  to  general  predictability  theory  and  the  limiting  error  growth  is  true 
irrespective  of  the  weather  regime,  being  observed  in  both  the  winter  and  the 
summer  patterns.  Though  not  shown,  a  similar  result  was  also  found  for  the 
500mb  RMS  geopotential  height  errors.  The  cause  of  this  trend  can  be 
explained  by  the  requirement  that  the  forecasts  at  set  intervals  (here  every  12 
hours)  meet  the  conditions  of  the  boundaries  provided  by  the  “truth”  thus 
eliminating  the  errors  which  may  have  propagated  in  from  the  boundaries  at 
longer  wavelengths.  This  result  is  supported  by  the  observation  that  at  the  outer 
domain  (108  km),  both  wind  speed  and  geopotential  height  error  growth  curves 
for  all  ten  case  runs  show  a  definitive  convergence  at  the  forecast  times  where 
the  boundary  conditions  were  interjected.  This  was  followed  by  a  divergence  as 
the  model  resumed  its  characteristic  deviation  in  solutions. 

Using  the  MM5  control  as  the  boundary  conditions  for  the  forecasts  was  of 
course  an  indication  of  the  lower  bound  of  the  error  growth  and  so  in  order  to 
provide  a  more  realistic  view  of  the  error  growth  in  operational  settings,  NOGAPS 
was  used  as  the  boundary  conditions  for  the  MM5  forecasts.  Figure  19  shows 
the  resulting  500mb  RMS  wind  speed  error  growth  when  the  NOGAPS  boundary 
conditions  were  used  for  the  winter  pattern.  As  shown,  the  character  of  the  error 
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is  significantly  different  than  the  forecasts  which  used  the  MM5  control  boundary 
conditions.  The  upward  trend  in  error  growth  is  more  representative  of 
predictability  theory  and  the  realistic  errors  that  can  be  introduced  by  inaccurate 
LBCs.  Figure  20  shows  the  average  RMS  wind  speed  error  growth  (or  decay) 
of  the  ten  random  solutions  for  the  winter  pattern.  One  might  think  of  this  plot  as 
the  ensemble  average  of  the  winter  pattern  forecast  error.  This  plot  shows  how 
significant  lateral  boundary  conditions  are  to  the  character  of  the  error  growth  (or 
decay).  Though  the  initial  error  is  very  similar,  the  error  growth  curves  diverge  at 
around  F06,  and  by  F24  the  average  of  the  solutions  between  the  two  sets  of 
boundary  conditions  show  almost  a  4  m/s  spread  in  wind  speed  error. 

D.  THE  EFFECT  OF  MODEL  SPIN  UP  ON  ERROR  GROWTH 

We  observed  that  in  the  forecast  runs  for  each  case,  the  model  required  a 
certain  amount  of  time  to  spin  up  the  mesoscale  structure  not  present  at  F00. 
This  was  particularly  evident  at  the  4km  domain  as  seen  in  Figures  8d  and  9d, 
where  no  mesoscale  structure  was  present  at  F00.  By  about  F06,  there  was  a 
considerable  spread  in  the  solutions  provided  by  each  random  case,  essentially 
defining  the  maximum  variance  in  the  error  for  the  forecast  cycle.  The 
requirement  that  the  model  “spin  up”  the  mesoscale  structure  in  the  forecast  by 
initializing  directly  from  a  synoptic  scale  model  analysis  or  forecast  is  often 
termed  a  “cold  start”  at  operational  numerical  forecasting  centers  (Hodur  1997). 
Monterossa  (1999)  documented  the  impact  of  cold  starts  on  a  mesoscale 
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forecast  showing  that  in  some  cases,  forecasts  were  actually  better  when  a  “cold 
start”  was  performed.  The  observable  error  growth  using  a  previous  6  hour 
forecast,  which  is  known  as  a  “warm  start,”  was  not  performed  during  this  study 
although  comparing  the  results  might  prove  interesting. 

Additionally,  almost  every  error  growth  curve  at  all  domains  for  both 
weather  regimes  and  boundary  conditions  showed  an  initial  downward  trend  in 
error,  which  is  also  attributable  to  the  effects  of  the  model  spin  up.  Recalling  the 
theoretical  NMFE  growth  curve  in  Figurel ,  this  initial  downward  trend  reaches 
an  error  minimum  before  the  forecast  error  begins  to  grow.  This  is  the  point  at 
which  the  model’s  initial  state  becomes  consistent  with  the  physical  equations 
that  govern  the  model’s  behavior.  Another  interesting  study  for  future  research 
would  involve  manipulating  the  initial  conditions  in  such  a  way  as  to  achieve  the 
lowest  error  minimum  for  the  longest  period  of  time. 


E.  WEATHER  REGIME  DEPENDENCIES 

1.  Kinetic  Energy 

As  expected,  there  also  appeared  to  be  a  weather  regime  dependence  in 
the  error  growth  characterized  by  smaller  error  values  for  the  summer  case  than 
for  the  winter  case.  The  reason  for  this  can  be  assumed  to  be  the  smaller 
amount  of  kinetic  energy  associated  with  the  summer  regime.  The  benefit  gained 
from  increasing  the  number  of  observations  was  also  less  distinct  in  the  summer 
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regime  than  in  the  winter  regime  for  the  same  reason.  It  should  be  noted, 
however,  that  the  relative  size  of  mesoscale  error  for  both  weather  patterns  when 
compared  to  the  mean  values  of  those  patterns  were  similar.  In  other  words,  the 
summer  pattern  was  not  any  more  predictable  than  the  winter  pattern  just 
because  the  magnitude  of  its  errors  were  less. 

2.  Unique  Mesoscale  structure 

We  noted  that  for  all  the  case  runs,  the  type  of  mesoscale  structure 
present  seemed  to  impact  the  character  of  the  error  growth  at  certain  times  in 
the  forecast  runs.  For  example,  in  the  winter  case  at  the  4km  domain,  there  was 
a  sharp  increase  in  500mb  RMS  wind  speed  error  at  FI  8  followed  by  a  sharp 
decrease  in  error.  As  mentioned  earlier  in  the  description  of  the  winter  weather 
pattern,  the  passage  of  the  cyclone’ s  cold  front  occurs  around  this  same  time. 
This  trait  is  also  reflected  in  the  500mb  wind  speed  errors  bias  (Figure  21),  which 
shifted  significantly  from  positive  pre-cold  front  to  negative  post-cold  front  (recall 
that  bias  estimates  were  performed  by  subtracting  the  random  case  from  the 
“control”  case).  For  the  summer  regime,  the  character  of  the  error  throughout 
the  forecast  cycle  was  fairly  constant  due  to  the  less  well  defined  mesoscale 
structure  of  this  pattern  (see  Figures  9[a-d]).  Furthermore,  there  was  no  jump  in 
error  for  the  summer  case  primarily  because  the  sea/land  breeze  feature  that 
was  present  is  thermally  forced  by  the  well  known  surface  boundary  and  is 
spatially  fixed  in  the  domain  throughout  the  forecast.  The  winter  frontal  structure 
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however,  is  forced  on  all  sides  throughout  the  column  and  moves  in  space  as  the 
forecast  develops.  Thus,  the  winter  pattern  showed  a  more  perceptible  jump  in 
error. 


F.  DOMAIN  SIZE  DEPENDENCIES 

There  was  also  a  difference  in  the  way  that  errors  grew  between  scale 
sizes.  In  general,  there  was  less  error  at  the  larger  domain  sizes  while  error 
magnitudes  increased  with  decreasing  scale  sizes,  reaching  lOm/s  for  some 
solutions  at  the  winter  pattern’s  4km  domain  (Figure  8d).  Again,  this  was  more 
evident  with  the  winter  pattern  than  in  the  summer  pattern.  This  result  combined 
with  the  larger  errors  at  the  smaller  scales  suggests  that  mesoscale  structure  is 
more  important  at  these  scales. 

Note  in  Figures  8d  and  9d  that  each  of  the  random  case  solutions  start 
with  the  same  error  value  at  F00  at  the  4km  domain.  This  is  due  to  the  fact  that 
no  observations  were  located  within  this  domain  size  at  initialization  and  thus 
each  case  was  initialized  with  the  same  NOGAPS  first  guess  fields.  After 
initialization,  the  error  growth  curves  depart  due  to  the  impact  of  the  observations 
at  the  larger  scales. 

Comparing  the  random  case  solution’s  error  growth  curves  with  the 
standard  deviation  about  the  mean  of  the  control  forecast  shows  that  model  skill 
decreases  with  scale.  While  all  the  model  solutions  show  skill  throughout  the 
entire  forecast  at  the  108  and  36km  domains,  some  of  the  solutions  begin  to  lose 
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skill  at  12km  in  the  winter  pattern.  At  the  4km  domain,  there  is  little  to  no  skill  at 
all,  regardless  of  weather  pattern  or  boundary  condition. 


G.  SYNOPTIC  SCALE  INFLUENCE  ON  THE  MESOSCALE 

One  of  our  hypotheses  stated  that  variation  in  the  observational  sampling 
pattern  at  the  synoptic  scale  leads  to  variation  in  analyses  that  directly  impact 
error  growth  and  predictability  in  mesoscale  forecasts.  Some  of  the  more 
interesting  results  we  found  throughout  the  course  of  this  work  dealt  with  how 
synoptic  scale  error  impacts  mesoscale  error. 

1 .  Synoptic  and  Mesoscale  Correlation 

Contrary  to  our  hypothesis  and  general  consensus  among  forecasters 
(Gunderson  1999),  there  was  not  a  great  amount  of  correlation  between  the 
errors  on  the  synoptic  scale  and  those  on  the  mesoscale.  Figure  22  shows  the 
correlation  scatter  plots  of  500mb  wind  speed  errors  between  the  108km  and 
4km  domains  for  both  weather  regimes  and  boundary  conditions.  Table  3  below 
summarizes  the  correlation  coefficients  between  the  synoptic  scale  (108km)  and 
mesoscale  (4km)  500mb  wind  speed  errors.  All  random  cases  and  all  forecast 
times  are  included  in  these  calculations. 
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1 0  cases  x  24  test  times  =  240  pts 

Winter 

Summer 

COAMPS  B.C. 

0.05694 

0.407778 

NOGAPS  B.C. 

0.47736 

-0.00923 

Table  3.  Correlation  Coefficients  between  synoptic  scale  (108km) 
and  mesoscale  (4km)  500mb  wind  speed  errors 
(all  cases,  all  forecast  times) 


It  is  evident  from  these  results  that  there  is  not  much  correlation  between 
the  synoptic  scale  and  mesoscale  errors  regardless  of  weather  regime  or 
boundary  condition.  Even  the  winter  pattern  using  NOGAPS  as  boundary 
conditions  which  has  the  greatest  amount  of  correlation  shows  a  rather  bi-modal 
distribution  on  its  scatter  plot  diagram.  These  results  must  be  somewhat 
tempered  since  the  RMS  errors  we  calculated  were  calculated  across  the  entire 
domain  for  each  domain  size.  Thus,  the  low  correlation  values  between  the 
108km  and  4km  scales  are  expected. 

2.  Scale  Separation  of  Error 

The  scale  specific  500mb  wind  speed  error  for  case  cl  48954  on 
980206/00zF12  is  shown  in  Table  4.  These  values  were  obtained  by  first 
projecting  the  three  larger  scale  fields  from  both  the  random  case  run  and  the 
MM5  “control”  onto  the  4km  grid  scale  (Figure  7).  We  then  created  the 
difference  fields  by  subtracting  the  scale  specific  fields  of  the  case  from  the 
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control  fields.  RMS  and  bias  errors  were  then  calculated  from  the  difference 
fields. 


RMS  (m/s) 

BIAS  (m/s) 

108km 

1.95  (46%) 

-1.41 

36km 

1.02  (23%) 

-0.35 

12km 

0.86  (20%) 

-0.05 

4km 

0.45  (11%) 

0.02 

Total 

4.28 

-1.79 

Table  4.  Scale  separation  of  500mb  wind  speed  error  for  980206/00zF12 
(case  cl  48954) 

As  can  be  seen  by  the  results  listed  in  Table  4  above,  the  majority  of  the 
total  error  at  the  smaller  scales  is  due  to  the  characteristic  error  of  the  synoptic 
scale.  The  summation  of  the  errors  at  each  scale  also  shows,  however,  that  our 
initial  assumption  about  the  independence  of  the  errors  at  each  scale  was  not 
completely  correct.  The  calculated  RMS  error  on  the  4km  domain  which  should 
be  a  sum  of  all  the  representative  scale  errors  was  2.5  m/s  while  the  summed 
value  above,  calculated  by  separating  out  the  representative  scales  and  then 
summing  them  is  4.28  m/s.  This  suggests  that  that  the  error  at  the  smaller 
scales  is  not  exactly  additive  and  we  have  not  accounted  for  all  of  the  scale 
interactions  actually  present.  This  fact  however,  does  not  change  the  fact  that 
the  largest  amount  of  error  at  the  4km  scale  is  still  dominated  by  108km  scale 
error  present  on  the  4km  domain. 
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IV.  DISCUSSION 


Further  examination  led  us  to  develop  some  possible  reasoning  behind 
the  results  we  achieved  from  our  experiments.  Though  limited  to  two  case 
studies,  the  analysis  of  these  results  have  shed  considerable  light  on  the 
character  of  numerical  model  error  on  the  mesoscale  and  the  limits  of 
predictability  on  these  scales.  The  discussion  in  this  section  will  thus  explain  the 
results  from  the  previous  section  in  more  detail  and  will  tie  key  concepts 
together. 


A.  CHARACTERISTIC  WAVELENGTHS 

On  the  outer  (108km)  domain,  the  model  error  has  a  characteristic 
wavelength  which  is  based  on  the  domain  size,  location,  and  the  distance 
between  the  observations.  Numerical  models  can  resolve  features  that  have 
wavelengths  of  2Ax  or  greater,  where  Ax  is  the  observational  spacing.  Thus,  a 
typical  observational  spacing  of  200km  for  instance,  will  allow  the  model  to 
resolve  wavelengths  on  the  order  of  400km  (Nuss  1998).  Domain  size  and 
location  is  also  important  when  considering  a  specific  weather  system  as  in  this 
study.  For  example,  in  the  winter  regime,  the  108km  domain  was  situated  in 
such  a  way  as  the  be  able  to  capture  the  entire  long  wave  trough  and  its 
associated  short  wave  pattern.  Thus,  error  associated  with  timing  and  location  of 
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that  trough  was  fairly  accurate  when  compared  to  a  domain  location  that  may 
have  only  captured  a  portion  of  the  trough. 

Many  of  the  findings  of  our  study  seemed  to  indicate  that  there  were 
characteristic  wavelengths  of  error  that  were  allowed  to  grow  while  other 
wavelengths  were  unresolvable.  The  effect  of  the  various  domain  sizes  as  well 
as  the  number  of  observations  on  the  error  growth  both  point  to  this  conclusion. 
The  value  to  which  the  error  asymptoted  at  F24  for  the  outer  two  domains  can  be 
thought  of  as  the  error  associated  with  the  minimum  resolvable  wavelength  at 
those  domain  sizes.  We  can  expect  this  value  to  change  with  different 
combinations  of  domain  size,  location,  and  observational  spacing.  For  example, 
increasing  the  number  of  observations  in  our  study  reduced  the  spacing  between 
the  observations  (Figure  4),  effectively  reducing  the  size  of  the  wavelengths  that 
could  be  resolved.  As  a  result,  the  magnitude  of  the  error  was  reduced. 


B.  LATERAL  BOUNDARY  CONDITIONS 

Related  to  the  characteristic  wavelength  of  the  error  due  to  domain  size 
and  location  is  the  understanding  that  the  different  lateral  boundary  conditions 
imposed  on  the  model  in  effect  limited  the  wavelengths  of  the  allowable  solutions 
at  the  largest  domain.  The  error  growth  at  this  domain  then  consists  only  of  the 
error  associated  with  the  scales  that  could  freely  grow  within  the  domain.  For 
this  reason,  when  the  MM5  “control”  forecast  was  used  as  the  boundary 
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condition,  the  error  for  all  cases  actually  decreased  with  forecast  time,  because 
the  conditions  at  the  boundary  had  to  be  met.  Of  course,  this  also  occurred 
when  NOGAPS  was  used  as  the  boundary  condition  but  since  the  NOGAPS 
conditions  didn’t  maintain  as  strict  a  control  over  the  true  mesoscale  structure  of 
the  system  as  the  “control”  conditions,  certain  wavelengths  that  were  fixed  under 
the  “control”  boundary  conditions  were  allowed  to  grow  under  the  NOGAPS 
boundary  conditions.  This  resulted  in  the  conventional  error  growth  that  we 
observed  in  Figures  8  and  9. 

The  significant  influence  of  the  lateral  boundary  conditions  on  the  outer 
domain  and  therefore  the  character  of  the  error  at  all  domains  is  further 
supported  by  the  results  of  the  scale  separation  experiment  we  performed. 
These  results  showed  that  the  error  at  all  domains  is  dominated  by  the  error  at 
the  largest  domain,  which  is  primarily  influenced  by  the  boundary  conditions. 

Recalling  the  result  of  the  reversal  of  the  error  at  FI 2  when  200 
observation  points  were  used  adds  further  insight  into  the  impact  of  the  boundary 
conditions.  Increasing  the  number  of  observations  from  100  to  200  points 
caused  the  dynamic  structure  of  the  field  to  be  better  defined,  making  it  become, 
in  essence,  more  sensitive  to  the  impact  of  the  boundary  conditions.  This,  in 
turn,  created  a  larger  error  when  compared  to  the  less  sensitive  100  point  case. 

The  sensitivity  of  a  particular  regime  to  the  boundary  conditions  is  also 
confirmed  when  we  compare  the  impact  of  the  boundary  conditions  on  the 
different  weather  regimes.  The  winter  pattern,  which  was  a  synoptically  forced 
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system,  was  particularly  sensitive  to  the  boundary  conditions  while  the  summer 
regime  was  not.  The  summer  pattern  on  the  other  hand  was  dominated  primarily 
by  local  forcing  and  had  a  less  well-defined  mesoscale  structure. 

It  is  clear  from  the  various  ways  in  which  we  examined  the  effect  of  the 
boundary  conditions  on  error  growth,  that  lateral  boundary  conditions  play  an 
important  role  in  the  character  of  error  at  all  scales.  Especially  for  the  case  of  a 
synoptically  forced  system  such  as  the  winter  pattern  in  our  study,  it  is  more 
important  to  have  accurate  boundary  conditions  at  the  outer  domain  than  to  have 
lower  initial  error  when  trying  to  achieve  the  best  mesoscale  forecast. 


C.  SYNOPTIC  SCALE  INFLUENCES 

One  of  the  difficulties  of  understanding  error  at  any  scale  is  being  able  to 
get  at  the  unique  error  associated  with  that  scale  while  still  maintaining  a  proper 
perspective  of  the  influences  other  scales  have  on  the  total  error.  Though  only 
performed  for  a  single  case,  the  findings  from  the  scale  separation  experiment 
showed  that  total  error  in  a  mesoscale  forecast  is  dominated  by  error  at  the 
largest  scales.  This  may  explain  why  common  mesoscale  forecasting 
experience  has  shown  that  even  though  errors  are  large  at  the  mesoscale, 
mesoscale  forecast  fields  look  reasonable  and  are  useful  tools  for  the  forecaster. 
Although  the  correlation  measurements  between  the  synoptic  and  the  mesoscale 
error  were  not  as  definitive  as  we  might  have  hoped,  we  must  consider  that  we 
did  not  correlate  the  exact  representation  of  the  synoptic  scale  error  on  the  4km 
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domain  with  the  error  unique  to  the  4km  scale.  Rather,  we  correlated  the  error 
across  the  entire  108km  domain  with  that  across  the  significantly  smaller  4km 
domain.  When  this  is  considered,  the  small  correlation  coefficients  were  a 
reasonable  result.  We  should  not  expect  that  there  would  be  a  great  deal  of 
correlation  between  error  across  so  large  a  domain  with  error  on  such  a  small 
domain.  The  understanding  the  forecaster  must  have  however,  is  that  synoptic 
scale  error  does  impact  the  error  at  the  mesoscale,  but  that  the  correlation  of 
such  error  is  only  significant  at  the  domain  size  of  interest.  Thus,  even  if  a  model 
seems  to  be  handling  the  synoptic  scale  pattern  well,  if  the  synoptic  scale  error  is 
large  in  the  location  of  the  mesoscale  nest,  there  will  be  a  great  deal  of  error 
associated  with  the  mesocale  forecast. 

Our  findings  with  the  scale  separation  experiment  also  showed  that  error 
at  each  scale  is  not  truly  separable.  The  error  at  a  specific  scale  is  more  likely  to 
be  the  integral  of  error  contributed  by  all  the  wavelengths  represented.  Since  our 
assumption  was  based  on  a  finite  differencing  approach  to  the  calculation  of  the 
total  error  based  only  on  the  scales  included  in  the  nested  grid,  our  results  did 
not  completely  match  up  with  our  expectations.  Furthermore,  even  though  the 
smaller  nested  domains  are  linked  to  the  larger  domains,  they  are  still  in  essence 
separate  forecasts.  Thus,  the  larger  scale  fields  we  projected  onto  the  4km  grid 
did  not  completely  correspond  at  every  grid  point. 
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D.  OBSERVATIONAL  SAMPLE  SIZE  AND  DISTRIBUTION 


The  random  distribution  of  observations  in  this  study  provided  unique 
insight  into  the  error  variance  associated  with  different  model  solutions.  The  fact 
that  our  findings  suggest  very  little  correlation  between  analysis  error  and  24 
hour  forecast  error  was  particularly  enlightening. 

It  was  clear  from  the  results  of  the  scale  separation  experiment  that  the 
spread  in  the  various  cases  at  the  smaller  domains  as  seen  in  Figures  8d  and  9d 
was  due  to  the  small  differences  at  synoptic  scales  exciting  particular  mesoscale 
frequencies.  Thus,  the  spread  is  in  essence  the  measure  of  the  sensitivity  of  the 
small  scale  error  to  synoptic  scale  differences.  For  our  study,  the  winter  case 
was  more  sensitive  most  likely  because  of  its  higher  kinetic  energy,  more  well- 
defined  mesoscale  structure,  and  the  fact  that  it  was  a  synoptically  forced 
system. 

Increasing  the  number  of  observations  from  100  to  200  points  at 
initialization  showed  that  more  points  does  not  always  equal  a  better  forecast, 
particularly  at  smaller  scales.  The  added  observations  must  define  realistic  and 
dynamically  significant  mesoscale  structure  in  order  to  provide  any  benefit. 


E.  WEATHER  REGIME  DEPENDENCIES 

It  was  clear  that  there  were  some  weather  regime  dependencies  on 
mesoscale  error  growth  and  predictability.  However,  the  weather  regimes  did  not 
play  as  great  a  role  in  influencing  the  error  growth  as  the  boundary  conditions 
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did.  In  general,  the  more  mesoscale  structure  and  kinetic  energy  a  system  has, 
the  more  likely  it  will  be  to  have  greater  magnitude  errors  and  the  more  likely  it  is 
to  be  influenced  significantly  by  the  accuracy  of  the  boundary  conditions. 
Weakly  forced  systems,  such  as  the  summer  pattern  in  this  study,  are  not  greatly 
influenced  by  error  at  the  boundaries.  Initial  error  in  the  summer  pattern  was 
high  and  error  remained  saturated  throughout  the  forecast  cycle,  particularly  at 
the  12km  and  4km  scales.  This  suggests  that  we  do  have  a  great  amount  of  skill 
at  these  scales,  even  at  initial  times  in  the  forecast. 

Though  we  did  not  address  specific  mesoscale  features  directly 
influenced  by  topography,  we  may  assume  that  the  magnitude  of  the  error 
associated  with  the  summer  pattern  was  less  than  that  of  the  winter  pattern  in 
part  because  most  of  the  mesoscale  structure  was  forced  by  the  surface.  This 
fact  requires  that  in  order  to  ensure  that  mesoscale  structure  will  be  correctly 
forced,  the  model’s  representation  of  the  topography  must  also  be  accurate. 
Current  mesoscale  model  topographies  smooth  out  many  significant  features  in 
addition  to  misrepresenting  elevations  (Monterrosa  1999),  making  the  calculation 
of  error  at  the  smallest  scales  even  more  complex.  Performing  a  study  of  the 
error  associated  only  with  the  representation  of  the  topography  would  be  a 
worthwhile  future  endeavor. 
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V.  CONCLUSIONS 


The  results  of  this  thesis  have  shown  that  although  there  have  been  some 
marked  improvements  in  mesoscale  forecasting,  there  are  still  some  questions 
with  regards  to  how  accurate  we  can  be  at  the  smallest  domain  sizes.  We 
showed  that  error  growth  is  affected  by  many  different  variables,  some  of  which 
we  can  control  and  others  which  we  cannot,  but  all  of  which  we  must  understand 
in  order  to  provide  accurate  forecasts.  Specifically,  we  showed  how  error  growth 
on  the  mesoscale  is  dependent  upon: 

•  Lateral  Boundary  Conditions 

•  Weather  regime 

•  Sample  size  and  distribution 

•  Synoptic  scale  error 

•  Domain  size 

For  these  specific  model  solutions,  we  also  showed  the  relationship  between: 

•  Analysis  error  and  forecast  error. 

•  Synoptic  scale  error  and  mesoscale  error. 

•  Unique  error  at  different  scales  through  scale  separation. 
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A.  MESOSCALE  CERTAINTIES  AND  UNCERTANTIES 


Many  of  our  initial  expectations  and  hypotheses  were  confirmed  by  the 
results  of  the  various  experiments  while  other  results  were  more  surprising. 
Some  results  actually  varied  from  previous  work  in  this  field.  For  instance,  unlike 
Anthes’  (1986)  results,  which  showed  that  72-hour  simulations  in  limited-area 
models  are  not  sensitive  to  random  uncertainties  in  initial  wind,  temperature,  and 
moisture  fields,  our  results  have  shown  that  even  in  a  24  hour  simulation,  error 
growth  does  occur  at  the  4km  scale. 

The  following  statements  reflect  the  summary  of  our  findings  and  help  to 
clarify  those  ideas  which  we  hope  will  add  to  the  understanding  of  mesoscale 
error  growth  and  predictability. 

1 .  Lateral  boundary  conditions  -  LBC’s  play  a  critical  role  in  defining 
how  error  grows,  particularly  for  weather  patterns  that  are  synoptically 
forced  such  as  the  winter  regime  in  this  study.  The  stronger  the 
synoptic  scale  forcing,  the  more  important  the  boundary  conditions 
become  in  interjecting  error  or  correcting  error  in  the  forecast. 

2.  Weather  patterns  -  For  weather  patterns  that  are  weakly  forced  such 
as  the  summer  regime  in  this  study,  error  growth  remains  small 
throughout  the  first  24  hours.  Initial  error  is  large  for  these  systems  at 
all  scales  and  becomes  quickly  saturated  throughout  the  forecast 
cycle. 
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3.  Forecast  skill  -  For  these  cases,  there  was  little  to  no  forecast  skill  at 
the  4km  domain  when  comparing  error  to  the  standard  deviation  about 
the  mean  of  the  control  field. 

4.  Synoptic  and  mesoscale  correlation  -  There  is  very  little  correlation 
between  synoptic  scale  error  and  mesoscale  error,  hence,  a  good 
synoptic  scale  forecast  does  not  guarantee  a  good  mesoscale 
forecast.  According  to  the  scale  separation  of  error,  however,  synoptic 
scale  error  dominates  the  error  at  the  mesoscale.  Thus,  a  poor 
synoptic  scale  forecast  will  also  result  in  a  poor  mesoscale  forecast, 
particularly  if  the  error  is  located  at  the  domain  of  interest. 

5.  Analysis  and  Forecast  error  correlation  -  While  correct  sampling  of 
the  atmosphere  through  sample  distribution  is  critical  to  ensuring  good 
analyses,  there  is  very  little  correlation  between  initial  error  and 
forecast  error.  Thus,  a  good  analysis  does  not  guarantee  a  good  24 
hour  forecast,  particularly  for  weakly  forced  weather  regimes. 

6.  Sample  size  -  Error  is  generally  reduced  when  more  observations 
are  used  at  initialization,  particularly  at  large  domain  sizes  for  weather 
regimes  that  are  synoptically  forced.  Also,  some  observations  are 
better  than  no  observations  when  calculating  error  growth.  Increasing 
the  number  of  observations  for  a  nested  grid,  can  in  some  cases 
increase  error  at  the  smallest  nest. 
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B.  FURTHER  RESEARCH 


The  unique  approach  of  this  study  and  the  relative  ease  of  its 
implementation  opened  up  a  vast  number  of  opportunities  for  further  research. 
The  advantage  of  using  numerical  experiments  in  a  controlled  “ideal” 
atmosphere  cannot  be  overlooked.  The  consistency  we  were  able  to  produce 
with  our  results  and  the  certainty  we  gained  by  comparing  the  model  experiments 
to  an  ideal  atmosphere  added  to  the  validity  of  this  research  approach.  The 
following  suggestions  represent  possibilities  for  further  research  and  study  in  the 
subject  of  mesoscale  model  error  and  predictability. 

1.  Vary  Weather  Regimes  and  Geographic  Locations 

Natural  atmospheric  variability  as  well  as  the  growth  rate  of  small  errors 
varies  with  geographic  locations  and  synoptic  pattern  (Anthes  1986).  While  this 
study  provided  a  good  representation  of  the  typical  weather  regimes  found  in  the 
winter  and  summer  off  the  west  coast  of  the  U.S.,  performing  this  study  for  other 
weather  patterns  in  other  geographic  locations  would  lend  more  credibility  to  our 
results.  This  is  especially  important  since  most  of  the  error  across  the  smaller 
domains  in  the  nested  model  were  linked  to  the  topographic  features  common  to 
the  West  Coast  of  the  U.S. 
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2.  Calculate  Error  Growth  Using  other  Model  Parameters 

While  the  error  growth  and  predictability  patterns  we  found  were 
consistent  for  the  two  model  parameters  we  studied  (wind  speed  and 
geopotential  heights),  this  may  not  be  the  case  for  all  variables.  Further  results 
could  easily  be  obtained  for  temperature  and  moisture  parameters  in  the  forecast 
model,  which  would  add  more  depth  to  the  results  we  have  shown. 

3.  Better  Experimental  Design 

Some  constraints  were  placed  on  this  study  due  to  choices  that  were 
made  early  on  and  better  experimental  design  would  lead  to  more  robust  results. 
The  most  useful  of  these  changes  would  be  to  represent  the  control  and 
experimental  forecasts  at  the  full  40  levels  within  the  MM5  model.  This  change 
would  result  in  more  accurate  error  growth  calculations.  In  addition,  we  could 
have  calculated  the  true  model  error  within  the  study  and  then  subtracted  it  out 
from  the  total  error. 

Another  design  change  would  include  adding  a  second  model  initialization 
method.  All  of  the  results  in  this  study  were  performed  with  a  model  initialized 
from  a  “cold”  start.  If  a  “warm”  start  initialization  was  also  performed,  we  could 
compare  the  resulting  error  growth  with  the  “cold”  start  results  and  draw  some 
conclusions  on  which  method  produced  better  forecasts. 
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Thirdly,  while  the  controlled  environment  of  this  study  afforded  great 
flexibility  in  design,  the  use  of  real  time  observations  to  calculate  error  would  help 
to  support  some  of  the  results  we  found. 

4.  Error  Growth  Compared  to  Synoptic  Forecast  Over  Time 

While  we  did  perform  a  forecast  experiment  by  using  only  the  synoptic 
scale  NOGAPS  fields  to  initialize  the  mesoscale  forecast,  it  would  certainly  be 
interesting  to  calculate  error  growth  statistics  for  a  completely  synoptic  scale 
forecast.  This  may  answer  the  question:  “Can  a  synoptic  scale  forecast  actually 
be  a  better  forecast  than  a  mesoscale  forecast?” 

5.  Use  of  Spectral  Analysis  to  Perform  Scale  Separation 

The  scale  separation  experiment  provided  some  interesting  results  which 
could  easily  be  repeated  for  different  weather  regimes  and  forecast  times. 
However,  since  our  results  showed  that  we  were  not  able  to  capture  the  error 
specific  to  all  wavelenghts,  a  spectral  analysis  of  the  error  might  be  a  good 
approach  in  performing  scale  separation  in  the  future. 

6.  Topography 

Topography  played  a  major  role  in  the  character  of  the  error  throughout 
this  study,  yet  the  model  topography  within  the  model  was  much  smoother  than 
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reality.  If  the  forecast  error  specifically  due  to  topographic  errors  was  able  to  be 
removed,  a  more  accurate  representation  of  error  growth  could  be  given. 
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APPENDIX  A.  FIGURES 


The  following  pages  of  figures  are  grouped  together  in  this  appendix  in 
order  to  facilitate  the  reading  of  this  work. 
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Figure  1 .  Theoretical  NMFE  growth  curves 
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Figure  2.  Domain  sizes  for  the  experiment’s  nested 
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Step  4.  Error  Growth  calculations 
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Figure  3.  Schematic  diagram  describing  the  design  of  the  ideal  atmosphere 
for  the  study. 
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Figure  4.  Random  distribution  of  100,200,  and  400  observations  for  case 
cl 48954. 
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Figure  5a.  MM5  control  FOO  forecast  for  winter  pattern.  Fields  are  500mb 
geopotential  hts.vorticity,  and  winds  (top)  and  surface  pressure, temperature, 
and  winds(bottom). 
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Figure  5b.  MM5  control  F12  forecast  for  winter  pattern.  Fields  are  500mb 
geopotential  hts,vorticity,  and  winds  (top)  and  surface  pressure,temperature, 
and  winds(bottom). 
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Figure  5c.  MM5  control  F24  forecast  for  winter  pattern.  Fields  are  500mb 
geopotential  hts.vorticity,  and  winds  (top)  and  surface  pressure, temperature, 
and  winds(bottom). 
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Figure  6a.  MM5  control  F00  forecast  for  summer  pattern.  Fields  are  500mb 
geopotential  hts.vorticity,  and  winds  (top)  and  surface  pressure, temperature, 
and  winds(bottom). 
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Figure  6b.  MM5  control  FI 2  forecast  for  summer  pattern.  Fields  are  500mb 
geopotential  hts,vorticity,  and  winds  (top)  and  surface  pressure, temperature, 
and  winds(bottom). 
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Figure  6c.  MM5  control  F24  forecast  for  summer  pattern.  Fields  are  500mb 
geopotential  hts.vorticity,  and  winds  (top)  and  surface  pressure,temperature, 
and  winds(bottom). 
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Figure  7.  108,  36, 12,  and  4km  scale  projections  on  the  4km  domain  grid  for 
the  winter  pattern  (MM5  control).  Fields  are  wind  barbs  and  isotachs  at  500mb. 
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Figure  8a.  500mb  wind  speed  error  growth  curves  on  108  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  8b.  500mb  wind  speed  error  growth  curves  on  36  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  8c.  500mb  wind  speed  error  growth  curves  on  12  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  8d.  500mb  wind  speed  error  growth  curves  on  4  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  9a.  500mb  wind  speed  error  growth  curves  on  108  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 


77 


500  mb  RMS  wind  speed  error 

summer  case  (36km  domain) 
NOGAPS  B.C. 


Cfl 

E 


0 

T5 

0 

0 

Q. 

0 


Forecast  Time 


-cl 48954  |j 
-  C333098  i: 
C475830 
C501118 
-C5783986 
C694725  I: 
C746294  f 

•  C750359 
C800231 
C97773 

•  sd  of  truth 


500  mb  RMS  wind  speed  error 

summer  case  (36km  domain) 

MM5  "control”  B.C. 


0 

1 


0 

*0 

I 

a 

03 

•o 

c 


10 

9 

8 

7 

6 

5 

4 

3 

2 


^  ^  #  <1^ 

Forecast  Time 


-<-0148954  • 
C333098  | 
C475830  j 

-  C501118  | 
— *—  C5783936 
— C694725  j 
•  -  C746294  j 
- C750369  ! 

-  C800231  1 
097773 

sd  of  truth 


Figure  9b.  500mb  wind  speed  error  growth  curves  on  36  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  1VIM5  control  (bottom). 
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Figure  9c.  500mb  wind  speed  error  growth  curves  on  12  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  9d.  500mb  wind  speed  error  growth  curves  on  4  km  domain  for  ten 
random  case  solutions  when  compared  to  the  MM5  control  forecast.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Correlation  between  F00  and  F24 
500mb  Wind  Speed  Errors  (108km  domain) 
summer  case ,  MM5  "control"  B.C. 
Correlation  coeff  =0.10863 
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Figure  10.  Correlation  diagrams  of  SOOmb  wind  speed  errors  between  F00 
and  F24  for  the  winter  (top)  and  summer  (bottom)  patterns  (108km  domain) 
using  MM5  control  (left)  and  NOGAPS  (right)  boundary  conditions. 
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Figure  1 1 .  Correlation  diagrams  of  500mb  wind  speed  errors  between  FOO 
and  F24  for  the  winter  (top)  and  summer  (bottom)  patterns  (36  km  domain) 
using  MM5  control  (left)  and  NOGAPS  (right)  boundary  conditions. 
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Figure  1 2.  Correlation  diagrams  of  500mb  wind  speed  errors  between  FOO 
and  F24  for  the  winter  (top)  and  summer  (bottom)  patterns  (12km  domain) 
using  MM5  control  (left)  and  NOGAPS  (right)  boundary  conditions. 
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Figure  13.  Correlation  diagrams  of  500mb  wind  speed  errors  between  FOO 
and  F24  for  the  winter  (top)  and  summer  (bottom)  patterns  (4  km  domain) 
using  MM5  control  (left)  and  NOGAPS  (right)  boundary  conditions. 
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Figure  14.  RMS  geopotential  height  error  at  analysis  for  winter  (left)  and 
summer  (right)  patterns. 
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Figure  15a.  500mb  wind  speed  error  growth  curves  on  108  km  domain  for 
case  cl 48954  with  100  and  200  point  initializations  in  the  winter  pattern. 
Boundary  conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  15b.  500mb  wind  speed  error  growth  curves  on  36  km  domain  for  case 
cl 48954  with  100  and  200  point  initializations.  Boundary  conditions  are 
NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  15c.  500mb  wind  speed  error  growth  curves  on  12  km  domain  for  case 
cl 48954  with  100  and  200  point  initializations.  Boundary  conditions  are 
NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  1 5d.  500mb  wind  speed  error  growth  curves  on  4  km  domain  for  case 
cl 48954  with  100  and  200  point  initiaiizations.  Boundary  conditions  are 
NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  16a.  500mb  wind  speed  error  growth  curves  on  108  km  domain  for 
case  C148954  with  100  and  200  point  initializations  for  summer  case. 
Boundary  conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  16b.  500mb  wind  speed  error  growth  curves  on  36  km  domain  for  case 
cl 48954  with  100  and  200  point  initializations  for  summer  case.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  16c.  500mb  wind  speed  error  growth  curves  on  12  km  domain  for  case 
C148954  with  100  and  200  point  initializations  for  summer  case.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  16d.  500mb  wind  speed  error  growth  curves  on  4  km  domain  for  case 
cl 48954  with  100  and  200  point  initializations  for  summer  case.  Boundary 
conditions  are  NOGAPS  (top)  and  MM5  control  (bottom). 
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Figure  1 7a.  500mb  wind  speed  error  growth  curves  on  108  km  (top)  and  36 
km  (bottom)  domains  for  case  cl 48954  with  100  point,  200  point,  and 
firstguess  only  initializations.  Plots  are  for  the  winter  case  with  MM5  “control” 
boundary  conditions. 
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Figure  17b.  500mb  wind  speed  error  growth  curves  on  12  km  (top)  and  4  km 
(bottom)  domains  for  case  cl  48954  with  100  point,  200  point,  and  firstguess 
only  initializations.  Plots  are  for  the  winter  case  with  MM5  “control”  boundary 
conditions. 
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Figure  18.  500mb  RMS  wind  speed  error  growth  curves  on  108  km  domain  for 
ten  random  case  solutions  in  the  winter  (top)  and  summer  (bottom)  patterns 
using  MM5  “control”  boundary  conditions. 
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Figure  19.  500mb  RMS  wind  speed  error  growth  curves  on  108  km  domain  for 
ten  random  case  solutions  in  the  winter  (top)  and  summer  (bottom)  patterns  - 
using  NOGAPS  boundary  conditions. 
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Figure  20.  “Ensemble”  average  of  500mb  RMS  wind  speed  error  growth 
curves  on  108  km  domain  for  all  ten  random  case  solutions  in  the  winter 
pattern  using  both  MM5  “control”  and  NOGAPS  boundary  conditions. 
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Figure  21 .  500mb  RMS  (top)  and  Bias  (bottom)  wind  speed  error  growth 
curves  on  4  km  domain  for  all  ten  random  case  solutions  in  the  winter  pattern 
using  MM5  “control”  boundary  conditions. 
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Figure  22.  Correlation  between  108km  and  4km  500mb  wind  speed  errors  for 
the  winter  (top)  and  summer  (bottom)  patterns  using  MM5  “control”  (left)  and 
NOGAPS  (right)  boundary  conditions 
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